基于级联卷积神经网络的手势特征提取方法

doi:10.11772/j. issn.1001-9081.2019122123

摘要/Abstract

摘要： 针对当前手势图像数据集不能均匀、全面地覆盖所有手势参数空间内的各种手势的问题，提出一种基于级联卷积神经网络的手势特征提取方法。该方法通过级联式模型，分层次地对高维度、高自由度的手势参数进行特征感知和提取。首先，将手腕角度参数作为手势参数的全局参数，进行划分和特征提取；然后，将手指角度参数作为局部参数，进行特征提取。为解决局部参数特征提取网络数量过多的问题，减少神经网络的数量和节约训练网络所需的时间与内存开销，采用多分支结构的神经网络模型，将五个手指的局部特征提取网络集成为一个整体。实验结果表明，所提方法在真实训练集上平均分类准确率达到 95. 13%，测试集平均准确率达到 54%，测试集准确率相较于全卷积神经网络的算法提高4. 76个百分点。

关键词: 手势主方向, 特征提取, 多分支结构, 级联卷积神经网络, 手势数据集

Abstract: Aiming at the problem that the existing data set can not cover all kinds of gestures in the parameter space evenly and comprehensively，a method of gesture feature extraction based on cascaded convolutional neural network was proposed. Gesture parameters with high dimension and freedom were detected and extracted by using cascaded model. Firstly，global parameters of wrist angles were divided and feature extraction was carried out；secondly，local parameters of finger angles were extracted respectively. In order to solve the problem of too many local parameter feature extraction networks，the number of neural networks was reduced and the time and memory cost of training networks were saved. A multi-branch neural network model was innovatively adopted to integrate the local feature extraction networks of five fingers into a whole. The experimental results show that the average classification accuracy of the proposed method is 95. 13% in training set；54% in test set，and 4. 76 percentage points higher than that of the algorithm based on full convolution neural network.

Key words: gesture main direction, feature extraction, multi-branch structure, cascaded convolutional neural network, gesture data set

中图分类号:

TP391.4

陈金龙瞿元昊杨明浩强保华唐仁俊朱庆杰 . 基于级联卷积神经网络的手势特征提取方法[J]. 计算机应用, DOI: 10.11772/j. issn.1001-9081.2019122123.

参考文献

[1]易靖国,程江华,库锡树.视觉手势识别综述[J].计算机科学, 2016(S1):103-108.(.[J]., 2016(S1):103-108.(in chinese)) [2]Tompson J, Stein M, Lecun Y, et al.Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks[C]// ACM, 2014:1-10.., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [3]Yuan S, Ye Q, Stenger B, et al.Bighand2. 2m benchmark: Hand pose dataset and state of the artanalysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017:4866-4874.., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [4]Tang D, Jin Chang H, Tejani A, et al.Latent regression forest: Structured estimation of 3d articulated hand posture[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2014:3786-3793.., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [5]Tang D, Yu T H, Kim T K.Real-time articulated hand pose estimation using semi-supervised transductive regression forests[C]//Proceedings of the IEEE international conference on computer vision. 2013: 3224-3231.., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [6]Sridhar S, Mueller F, Zollh?fer M, et al.Real-time joint tracking of a hand manipulating an object from rgb-d input[C]//European Conference on Computer Vision. Springer, Cham, 2016: 294-310.., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [7]王西颖, 戴国忠, 张习文等.基于HMM-FNN模型的复杂动态手势识别[J].软件学报, 2008(09):2302-2312.(.[J]., 2008(09):2302-2312.(in chinese)) [8]王西颖, 张习文, 戴国忠.一种面向实时交互的变形手势跟踪方法[J].软件学报, 2007(10):2423-2433.(.[J]., 2007(10):2423-2433.(in chinese)) [9]王红霞, 王坤.基于加锁机制的静态手势识别方法[J].计算机应用, 2016(07):1959-1964.(.[J]., 2016(07):1959-1964.(in chinese)) [10]尚常军, 丁瑞.基于曲率局部二值模式的深度图像手势特征提取[J].计算机应用, 2016(10):2885-2889.(.[J]., 2016(10):2885-2889.(in chinese)) [11]?aric M.Libhand: A library for hand articulation[J].Version 0.9, 2011(无):无-无.(.[J]., 2011(无):无-无.(in chinese))[J]., :-.(.[J]., ():-.(in chinese)) [12]Zimmermann C, Brox T.Learning to estimate 3d hand pose from single rgb images[C]//Proceedings., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [13]Sinha A, Choi C, Ramani K.Deephand: Robust hand pose estimation by completing a matrix imputedwith deep features[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 4150-4158.., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [14]Krizhevsky A, Sutskever I, Hinton G E.Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105.., :-.(.., :-.(in chinese))., ():-.(.., ():-.(in chinese)) [15]Srivastava N, Hinton G, Krizhevsky A, et al.Dropout: a simple way to prevent neural networks fromoverfitting[J]., 2014(1):1929-1958.(.[J]., 2014(1):1929-1958.(in chinese)) [16]Ioffe S, Szegedy C.Batch normalization: Accelerating deep network training by reducing internal covariate shift[J]. arXiv prep.[J].rXiv:1502.03167, 2015., rint:-.(.[J]., rint:-.(in chinese))[J]., ():-.(.[J]., ():-.(in chinese))